System and method for subjective property parameter determination

ABSTRACT

In variants, the method for subjective property scoring can include determining an objective score for a subjective characteristic of a property using a model trained using subjective property rankings.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.63/302,287 filed 24 Jan. 2022, which is incorporated in its entirety bythis reference.

TECHNICAL FIELD

This invention relates generally to the property appearance field, andmore specifically to a new and useful method and system for propertyappearance analysis.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a flowchart representation of a variant of the method.

FIGS. 2A and 2B are schematic representations of variants of the system.

FIG. 3 is a schematic representation of a variant of the method.

FIG. 4 is an illustrative example of a variant of the method.

FIG. 5 is an illustrative example of a second variant of the method.

FIG. 6 is an illustrative example of a third variant of the method.

FIG. 7 is a schematic representation of an example of the method,wherein the model is trained to determine the score using a differentinput modality than that used to determine the label (e.g., ranking).

FIG. 8 is an illustrative example of determining a label for a subjectpair.

FIG. 9 is a schematic representation of determining a subject's scoreusing the trained model.

FIGS. 10A, 10B, 10C, and 10D are schematic representations of a first,second, third, and fourth example of determining a subject's score usingthe trained model, respectively.

FIG. 11 is an illustrative example of determining a score for a subjectand presenting said score.

DETAILED DESCRIPTION

The following description of embodiments of the invention is notintended to limit the invention to these embodiments, but rather toenable any person skilled in the art to make and use this invention.

1. Overview

As shown in FIG. 1 , the method for property appearance analysis caninclude: determining a subject comparison group S100, determiningsubject information for the subject comparison group S200, determining alabel for the subject comparison group S300, and training a model basedon the label S400. The method can additionally or alternatively includedetermining a test subject S450, determining test information for thetest subject S500 and determining a score for the test subject S600.However, the method can additionally and/or alternatively include anyother suitable elements.

The method functions to determine an objective score for a subjectivecharacteristic of a subject. For example, the method can be used toscore the curb appeal of a house, the attractiveness of a housinginterior, the attractiveness of a landscape or yard, the appeal of aneighborhood, the condition of a roof, the health of a tree, and/orotherwise used. The objective score can be presented directly to a user(e.g., in an MLS™ application), be used as an input into a downstreammodel (e.g., an automated valuation model), and/or otherwise used.

2. Technical Advantages

Variants of the technology for property analysis can confer severalbenefits over conventional systems and benefits.

First, determining an objective score for a subjective characteristic(e.g., appeal, preferences, etc.) is incredibly difficult, particularlybecause humans inconsistently assign objective values to subjectivecharacteristics. However, the inventors have discovered that humans canconsistently evaluate subjective comparisons. For example, humans areinconsistent when rating the visual appeal of a house on a scale from1-10, but are relatively consistent when ranking houses based on whichhouse is more appealing than another. Variants of the technologyleverage this finding by training a model to determine an objectivescore (e.g., rating) for each property that is indicative of whether theproperty would rank higher or lower than a peer. This provides ataxonomy-free, standardized, objective score that describes thesubjective characteristic (e.g., level of attractiveness) of thesubject. In another example, conventional roof condition determinationmethods cannot accurately predict a roofs true condition, sinceconventional methods rely on single-timepoint data, and the roofcondition is a product of accumulated different forces acting upon theroof over time. While humans inconsistently assign roof condition labelsto roofs, they are generally consistent when ranking the condition ofone roof over another. Variants of the technology leverage this findingby using the relative roof condition rankings to train a model todetermine an objective roof condition score (e.g., rating) for eachproperty that is indicative of whether the roof would rank higher orlower than another property roof.

Second, variants of the technology can increase analysis accuracy byconsidering the objective score for the subjective characteristic. Forexample, the objective score can be used to improve the valuationaccuracy output by automated valuation models when used as an input,since the automated valuation models often suffer from valuation errordue to subjective characteristics that are not fully captured by theother objective inputs.

Third, variants of the technology can enable the objective score to bedetermined from other property parameters (e.g., other descriptionmodalities) that are more difficult for humans to compare. For example,the relative ranking between different properties can be determinedbased on images or videos of the respective property (e.g., exampleshown in FIG. 8 ), but the model can be trained to determine theobjective scores for each property based on the respective propertydescriptions, property attributes (e.g., beds, baths, year built, ownerdemographic, rental history, etc.), or other data (e.g., example shownin FIG. 7 ). This can increase the number of properties that can beobjectively scored, beyond those that have already been imaged.

However, the technology can confer any other suitable benefits.

3. Examples

In illustrative examples, the method can include: determining a subjectpair from a training subject set, wherein the subject pair includesdifferent subjects; determining an image set for each subject of thesubject pair, wherein the two image sets share the same image parameters(e.g., image quality, image size, scene class, perspective, etc.),manually determining a relative ranking (e.g., label) for the subjectpair (e.g., which subject is preferred) based on a subjective comparisonbetween the two image sets; optionally repeating the process for othersubject pairs to determine an overall relative ranking for each trainingsubject within the training subject set; and/or optionally determining arating for each subject based on the respective relative ranking.

In a first specific example, the score can be the subject's rating,wherein a model is trained to predict a subject's rating based on therespective subject information (e.g., image set) (e.g., illustrativeexample shown in FIG. 10A). In a first example, the model can be trainedusing the training subject's rating as a training target (e.g.,illustrative example shown in FIG. 4 ). In a second example, the modelcan be trained by predicting a first and second training subject'srating or other score using the model, wherein the first and secondtraining subject's rating then can be used to determine a predictedrelative ranking between the first and second training subject (e.g.,win/lose; preferred/unpreferred, etc.). The model is then trained on acomparison between the predicted relative ranking and the actualrelative ranking between the first and second training subjects (e.g.,illustrative example shown in FIG. 3 ).

In a second specific example, the score can be a rating bin or cluster,wherein the training subjects' ratings are binned into bins (e.g., 1-10)or clusters (e.g., with descriptive labels), and the model is trained topredict the subject's bin based on the respective image set (e.g.,illustrative examples shown in FIG. 6 , FIG. 10B, and FIG. 10C).Additionally or alternatively, the model can be trained to predict thesubject's rating (e.g., as in the first specific example), which is thenbinned into a predetermined bin or cluster.

In a third specific example, the score can be the subject's ranking(e.g., within the training population), wherein a model is trained todetermine a subject's rank (e.g., how a subject would be ranked relativeto the training population). An illustrative example is shown in FIG.10D.

The method can additionally or alternatively include: determining animage set for a subject identified in a request; and determining a scorefor the subject using the test image set and the trained model. Inexamples, the subjects can be properties (e.g., houses, parcels, etc.)that are up for sale, and the images can be obtained from a real estatelisting service. However, the model can be otherwise trained, and/or thescore can be otherwise determined.

4. Method

As shown in FIG. 1 , the method for property appearance analysis caninclude: determining a subject comparison group S100, determiningsubject information for the subject comparison group S200, determining alabel for the subject comparison group S300, and training a model basedon the label S400. The method can additionally or alternatively includedetermining a test subject S450, determining test information for thetest subject S500 and determining a score for the test subject S600.However, the method can additionally and/or alternatively include anyother suitable elements.

The method functions to train a model configured to output an objectivescore for a subjective characteristic (e.g., attractiveness, sentiment,appeal, preference, condition, etc.) of a subject given a measurement ofthe subject. Additionally or alternatively, the method can be used todetermine an objective score for a subjective characteristic of thesubject.

The method can be performed for one subjective characteristic, multiplecharacteristics, and/or any other suitable number of characteristics.The subjective characteristic(s) can be attractiveness, sentiment,appeal, preference, and/or any other suitable subjective characteristic.Examples of subjective characteristics include: attractiveness of theexterior of a subject (e.g., curb appeal), attractiveness of theinterior of a subject (e.g., kitchen inside a house), attractiveness oflandscaping surround a subject, attractiveness of subjects within aradius of a subject, and/or any other suitable subjectivecharacteristic.

All or portions of the method can be performed: in response to a requestfrom an endpoint, before receipt of a request, and/or any other suitabletime. The method can be performed for: all subjects within a subject set(e.g., all properties appearing in a measurement, etc.), a singlesubject (e.g., a requested property), and/or any other suitable set ofsubjects. One or more instances of the method can be repeated fordifferent subjects, different subjective characteristics, timeframes,perspectives, and/or otherwise repeated.

The method can be performed using one or more: subjects, labelsassociated with subject sets (e.g., subject comparison groups), andobjective scores, but can additionally or alternatively be performedusing any other suitable set of entities and/or data objects.

The subjects function as the entities for which objective scores(representing subjective characteristics) are determined. Each subjectcan be associated with one or more subjective characteristics. Thesubjective characteristics are preferably characteristics that aredifficult for humans to consistently rate, are based on personalfeelings, tastes, or opinions, and/or are otherwise defined. Thesubjective characteristic(s) can be attractiveness, sentiment, appeal,preference, condition, and/or any other suitable subjectivecharacteristic. Examples of subjective characteristics include:attractiveness of the exterior of a subject (e.g., curb appeal),viewshed appeal, attractiveness of the interior of a subject (e.g.,kitchen inside a house), attractiveness of landscaping surround asubject, attractiveness of subjects within a radius of a subject,condition of a roof, and/or any other suitable subjectivecharacteristic. The method can be used to determine one or moresubjective characteristics of a subject.

The subject(s) can be a property, a product, and/or any other suitablesubject. A property can be: real property (e.g., real estate, etc.), apoint of interest, a geographic region (e.g., a neighborhood), alandmark, a built structure (e.g., a house, condominium, warehouse,deck, etc.), a component of a built structure (e.g., a roof, a side of abuilt structure, etc.), a parcel, a portion of a parcel (e.g., a yard, abackyard, etc.), a physical structure (e.g., a pool, a statue, a deck,etc.), vegetation (e.g., a tree, a garden, etc.), a scene, any othersuitable object within a geographic region, and/or any other suitablesubject. Types of properties may include residential properties (e.g.,single-family home, multi-family home, apartment building, condominium,etc.), commercial properties (e.g., industrial center, forest land,farmland, quarry, retail, etc.), mixed-use properties, and/or any othersuitable property class. The subject can be identified by a subjectidentifier (e.g., a property identifier, such as an address, a lotnumber, parcel number, etc.), by a geographic region identifier (e.g.,latitude/longitude coordinates), not be associated with an identifier,and/or otherwise identified.

Each subject can be associated with a set of subject information. Thesubject information can be static (e.g., remain constant over athreshold period of time) or variable (e.g., vary over time). Thesubject information can be associated with: a time (e.g., a generationtime, a valid duration, etc.), a source (e.g., the information source),an accuracy or error, and/or any other suitable metadata. The subjectinformation is preferably specific to the subject, but can additionallyor alternatively be from other subjects (e.g., neighboring properties,other subjects sharing one or more attributes with the subject).

The subject information can include: measurements, measurement parametervalues, descriptions, auxiliary data, subject attributes, and/or anyother suitable information about the subject. The subject informationcan be sampled, retrieved from a third party (e.g., example shown inFIG. 2B), generated, and/or otherwise obtained.

The measurements function to measure an aspect about the subject. Eachmeasurement preferably depicts or is associated with the respectivesubject, but can alternatively not depict or not be associated with therespective subject. The measurements are preferably appearancemeasurements, but can additionally or alternatively be geometricmeasurements, acoustic measurements, and/or other measurements. Themeasurements can include: remote measurements (e.g., aerial imagery,satellite imagery, balloon imagery, drone imagery, etc.), local oron-site measurements (e.g., sampled by a user, streetside measurements,etc.), and/or sampled at any other proximity to the property. Themeasurements can depict one or more subjects. The measurements can be:top-down measurements (e.g., nadir measurements, panoptic measurements,etc.), side measurements (e.g., elevation views, street measurements,etc.), angled and/or oblique measurements (e.g., at an angle tovertical, orthographic measurements, isometric views, etc.), and/orsampled from any other pose or angle relative to the property. Themeasurement can be an image (e.g., 2D image, MLS™ image, etc.), a video,an audio, a digital surface model, a virtual model, a viewshedrepresentation, a point cloud, other imagery, and/or any other suitablemeasurement. Images can include oblique imagery (e.g., of a builtstructure, a street view image, etc.), aerial imagery, imagery of asubject's surroundings, exterior imagery (e.g., property interior),interior imagery, and/or any imagery. The measurements can depict theproperty exterior, the property interior, a property component, and/orany other view of the subject.

The measurements can be received as part of a user request, retrievedfrom a database, determined using other data (e.g., segmented from animage, generated from a set of images, etc.), synthetically determined,and/or otherwise determined.

Measurements can be associated with one or more measurement parametervalues. Measurement parameter values can include: scene class (e.g.,interior scene measurements, exterior scene measurements, etc.),perspective relative to the subject (e.g., front elevation, top planarview, front view, side view, etc.), pose relative to the subject,provider (e.g., vendor), format (e.g., JPEG, TIFF, PDF, RAW, etc.),modality (e.g., RBG camera, point cloud, etc.), season, measurementtime, measurement quality (e.g., pixel density, graniness, noise,resolution, zoom, etc.), measurement date, time of day, measurementlocation (e.g., latitude/longitude coordinates, position relative tosubject, etc.), and/or any other suitable contextual parameters. Invariants, when measurements of different subjects are used to determinethe labels (e.g., presented to a rater for rating), the measurementspreferably share at least one or more measurement parameter values(e.g., same quality, same resolution, same perspective, etc.);alternatively, the measurements can have different measurement parametervalues. The measurements used during training and runtime preferablyshare measurement parameter values, but can alternatively have differentmeasurement parameter values.

The subject information can include subject descriptions. The subjectdescription can be: a written description (e.g., a text description), anaudio description, and/or in any other suitable format. The subjectdescription is preferably verbal but can alternatively be nonverbal.Examples of subject descriptions can include: listing descriptions(e.g., from a realtor, listing agent, etc.), property disclosures,inspection reports, permit data, appraisal reports, and/or any othertext based description of a subject.

The subject information can include auxiliary data. Examples ofauxiliary data can include property descriptions, permit data, insuranceloss data, inspection data, appraisal data, broker price opinion data,property valuations, property attribute and/or component data (e.g.,values), and/or any other suitable data. The subject information caninclude subject attributes (e.g., subject parameter values), whichfunction to represent one or more aspects of a given subject. Thesubject attributes can be semantic, quantitative, qualitative, and/orotherwise describe the subject. Each subject can be associated with itsown set of subject attributes, and/or share subject attributes withother subjects. As used herein, subject attributes can refer to theattribute parameter (e.g., the variable) and/or the attribute value(e.g., value bound to the variable for the subject).

Subject attributes can include: subject class (e.g., house, physicalstructure, vegetation, property segment, etc.), subject subclass (e.g.,single-family house, multi-family house, apartment, condominium,commercial, mixed-use, etc.), location (e.g., neighborhood, ZIP code,etc.), location type (e.g., suburban neighborhood, urban neighborhood,rural, etc.), viewshed (e.g., lake view, mountain view, terrestrialview, adversarial view, etc.), built feature values (e.g., roof slope,roof rating, roof material, etc.), record attributes (e.g., number ofbed and baths, construction year, square footage, parcel area, etc.),condition attributes, semantic attributes (e.g., “turn key”, “move-inready”, “poor condition”, “walkable”, “popular”, “small”, any othertext-based descriptors, etc.), property values (e.g., subject propertyvalue, neighboring property value, etc.), risk asset scores (e.g., assetscore indicating risk of flooding, hail, wildfire, wind, house fire,etc.), vegetation parameters (e.g., coverage, density, setback, locationwithin one or more zones relative to the property), and/or any othersuitable set of attributes.

Subject attributes can be determined from and/or include subjectmeasurements, permit data, insurance loss data, inspection data,appraisal data, broker price opinion data, property valuations, propertyattribute and/or component data (e.g., values), and/or otherinformation. Subject attributes can be determined from governmentrecords, extracted from property measurements, and/or otherwisedetermined. Subject attributes can be determined based on subjectinformation for the subject itself, other subjects (e.g., neighboringproperties), and/or any other set of subjects. Subject attributes can beautomatically determined, manually determined, and/or otherwisedetermined. The subject attributes can be extracted using a model (e.g.,an NLP model, a CNN, a DNN, etc.) trained to identify keywords, trainedto classify or detect whether a subject attribute appears within theproperty information, and/or otherwise trained.

In variants subject attributes can be determined using one or more ofthe methods disclosed in: U.S. Pat. No. 10,311,302 issued Jun. 4, 2019,U.S. Pat. No. 11,222,426 issued Jan. 11, 2022, U.S. Pat. No. 11,367,265issued Jun. 21, 2022, U.S. application Ser. No. 17/870,279 filed 21 Jul.2022, U.S. application Ser. No. 17/858,422 filed 6 Jul. 2022, U.S.application Ser. No. 17/981,903 filed 7 Nov. 2022, U.S. application Ser.No. 17/968,662 filed 18 Oct. 2022, U.S. application Ser. No. 17/841,981filed 6 Jun. 2022, and U.S. application Ser. No. 18/074,295 filed 2 Dec.2022, all of which are incorporated herein in their entireties by thisreference. However, the subject attributes can be otherwise determined.

However, the subject information can include any other suitableinformation about the subject, and/or be otherwise determined.

Subjects can be grouped into one or more subject comparison groups.Subject comparison groups function as the entities for which asubjective comparison between multiple subjects can be determined (e.g.,labels are determined for the entire comparison group and/or betweenmembers of the comparison group). Each subject comparison grouppreferably includes two subjects (e.g., a subject comparison pair), butcan additionally and/or alternatively include three or more subjects, orinclude a single subject. Each subject comparison group can include aunique set of subjects, or alternatively multiple subject comparisongroups can include one or more subjects in common (e.g., the comparisongroups can be overlapping or disjoint). Each subject comparison grouppreferably contains the same number of subjects, but alternatively thesizes of subject comparison groups can vary across groups. Each subjectcomparison group preferably includes different subjects (e.g., house Aand house B), but can additionally and/or alternatively include the samesubject (e.g., house A and house A from different perspectives, house Aand house A at different points in time, house A and house A with aremodel, etc.). The subjects within each subject comparison group canhave the same subject attribute values and/or different subjectattribute values.

Subjects can optionally be determined from a subject set (e.g., atraining subject set, a test subject set, etc.). Subject sets functionas a group of subjects from which subjects can be determined for thepurposes of the method (e.g., a set of available subjects to split intotest and training subsets, a subject set from which comparison groupscan be determined, a target subject set for analysis, etc.). A subjectset can include one or more subjects. The subject set can be determinedfor any step of the method (e.g., for determining subject comparisongroups, for determining labels, as input for training the model, as atarget test subject, etc.). The subject set can be limited by one ormore subject attributes (e.g., only include single family homes, onlyinclude subjects from a single neighborhood), or can be unlimited.

The labels are preferably subjective characterizations of the subjects,but can alternatively be objective characterizations of the subjects.The labels are preferably indicative of relative rankings of differentsubjects, but can alternatively be used to infer the relative rankings(e.g., sentiment analysis is used to determine which property a labelerprefers based on the descriptions provided by the labeler). Eachsubjective characteristic can be associated with one or more labels. Alabel preferably indicates a winner for a subject comparison group basedon a subjective attribute (e.g., curb appeal). Alternatively, a labelcan indicate a loser, a tie, an order of preference within a subjectcomparison group, and/or other information comparing the subjects withina subject comparison group. Each label is preferably binary (e.g., winsand loses, 0 and 1, etc.), but can alternatively be non-binary (e.g.,multi-subject ranking). Each label can be a numerical label (e.g., 0, 1,2, etc.), a categorical label (e.g., wins, loses, ties, more appealing,less appealing, better, much better, etc.), and/or any other suitablelabel type. Each label is preferably associated with a comparisonbetween a set of subjects (e.g., subjects within a subject comparisongroup). However, the labels can be associated with individual subjects.The label preferably represents the relative ranking of the subjectswithin the set (e.g., which subject is preferred, which subject'ssubjective characteristic is higher or lower than the remainder of thesubjects, etc.); however, the label can represent a rating (e.g.,score), a classification, and/or any other suitable information. Thelabel can be determined: by a user; by inferring the label based ondescriptions or other subject information; and/or otherwise determined.The user can determine the label based on subject measurements (e.g.,images and/or video presented to the user on a user interface; exampleshown in FIG. 2B), by visiting the subjects (e.g., physically visitingthe property and labeling the subjects based on the onsite visit),and/or otherwise determine the label.

The objective score functions to provide an objective measure of asubjective characteristic. Each subjective characteristic is preferablyassociated with its own set of objective scores; alternatively,objective scores can be shared between subjective characteristics. Theobjective scores can be absolute (e.g., wherein one subject measurementmaps to one objective score), or alternatively relative (e.g.,regionally dependent, relevant to a subject attribute value, a measureof how the subject rates relative to the training property set, etc.).The objective score can be a numerical score (e.g., 100, 500, 2500,etc.), a classification (e.g., “appealing”, “not appealing”), acategorical variable (e.g., a whole number contained within a range; alabel such as “high appeal”, “moderate appeal, “low appeal”, etc.;etc.), and/or other objective metrics. The objective score can becontinuous, discrete, and/or otherwise characterized. The objectivescore is preferably quantitative, but can alternatively be qualitative.The objective scores for the subjects are preferably determined on thesame scale (e.g., such that the objective scores for two subjects can becompared against each other), but can alternatively be determined ondifferent scales. The objective scores can be normalized to apredetermined scale (e.g., converted to a scale of 1-10), binned intopredetermined classifications, provided as raw scores, and/or otherwisemodified or unmodified. In an example, the objective score can be acategorical variable value that reflects a relative position of asubject and/or subject measurement within a rank-based distribution ofthe set of ranked subjects and/or subject measurements.

The objective score can be: a rating (e.g., determined from the labelsor rankings), a score indicative of a ranking, a bin (e.g., wherein eachbin encompasses a set of rating values), a cluster (e.g., encompassing aset of rating values, encompassing a permutation of values for differentsubjective characteristic ratings, etc.), a ranking, and/or be any othersuitable score. For example, the objective score can be a rating (e.g.,Elo rating, Gliko rating, Harkness rating, etc.) indicative of how thesubject ranks (e.g., subjectively ranks) against all other consideredsubjects.

The objective score can be determined by the scoring model, by a ratingmodel (e.g., rating algorithm, such as the Elo rating algorithm, Glickorating algorithm, etc.), by a binning or clustering model, and/or by anyother suitable system. The objective score can be predicted, inferred,calculated, and/or otherwise determined.

However, the method can be performed using any other suitable set ofentities and/or data objects.

As shown in FIG. 2A, variants of the method can be performed using asystem 100 including one or more: scoring models, rating models,discretization models, and/or other models. The models function totransform information from one modality into a different modality,and/or perform other functions.

The models can be or include: neural networks (e.g., CNN, DNN, etc.), anequation (e.g., weighted equations), regression (e.g., leverageregression), classification (e.g., binary classifiers, multiclassclassifiers, semantic segmentation models, instance-based segmentationmodels, etc.), segmentation algorithms (e.g., neural networks, such asCNN based algorithms, thresholding algorithms, clustering algorithms,etc.), rules, heuristics (e.g., inferring the number of stories of aproperty based on the height of a property), instance-based methods(e.g., nearest neighbor), regularization methods (e.g., ridgeregression), decision trees, Bayesian methods (e.g., Naïve Bayes,Markov, etc.), kernel methods, statistical methods (e.g., probability),deterministics, support vectors, genetic programs, isolation forests,robust random cut forest, clustering, selection and/or retrieval (e.g.,from a database and/or library), comparison models (e.g., vectorcomparison, image comparison, etc.), object detectors (e.g., CNN basedalgorithms, such as Region-CNN, fast RCNN, faster R-CNN, YOLO,SSD—Single Shot MultiBox Detector, R-FCN, etc.; feed forward networks,transformer networks, generative algorithms (e.g., diffusion models,GANs, etc.), and/or other neural network algorithms), key pointextraction, SIFT, any computer vision and/or machine learning method(e.g., CV/ML extraction methods), and/or any other suitable model ormethodology.

The models can be trained using: self-supervised learning,semi-supervised learning, supervised learning, unsupervised learning,reinforcement learning, transfer learning, Bayesian optimization,positive-unlabeled learning, using backpropagation methods, and/orotherwise learned. The model can be learned or trained on: labeled data(e.g., data labeled with the target label), unlabeled data, positivetraining sets (e.g., a set of data with true positive labels, negativetraining sets (e.g., a set of data with true negative labels), and/orany other suitable set of data.

The scoring model functions to determine an objective score for asubject (e.g., property). The scoring model is preferably a machinelearning model, such as a neural network (e.g., CNN, RNN, etc.) or aclassical model, but can alternatively be any other suitable model. Thesystem can include one or more models. The scoring model can be specificto a subject, a subject class (e.g., house, physical structure, etc.), asubject subclass (e.g., single-family house, multi-family house, etc.),a subjective characteristic (e.g., appeal, attractiveness), a location(e.g., by street, by town, by city, by county, by state, by country, byZIP code, etc.), a location type (e.g., suburban neighborhood, urbanneighborhood, rural neighborhood, etc.), a perspective (e.g., exterior,interior, front view, back view, etc.), a measurement quality (e.g.,resolution, pixel density, etc.), a metadata value (e.g., a informationmodality, a provider, a perspective, etc.), rating method, an end user(e.g., customer; wherein the scoring model can be tuned using labelsreceived from the end user), and/or be otherwise specific. Additionally,and/or alternatively, the model can be generic across subjects, subjectclasses, subject subclasses, subjective characteristics, locations,location types, metadata values, and/or be otherwise generic.

The scoring model can determine (e.g., predict, infer, calculate, lookup, etc.) an objective score for a subject based on the subject'sinformation (e.g., measurements, parameters, etc.). The scoring modelpreferably determines the objective score based on measurements of thesubject (e.g., images, videos, depth information, etc.), but canadditionally or alternatively determine the objective score based onsubject attribute values (e.g., property attributes), subjectdescriptions, and/or other information.

The scoring model is preferably generated (e.g., trained) using thelabels (e.g., ranking data) for different subject sets, but can begenerated using other information. In a first variant, the scoring modelis trained to predict a rating for each of a set of training subjects(e.g., training properties), wherein the rating for each trainingsubject is determined based on a label associated with a subjectcomparison group that includes the training subject. In a secondvariant, the training subject population is discretized into bins,clusters, or categorical variable values based on the respectiveratings, wherein the model is trained to predict the rating. In a thirdvariant, the model is trained to predict an objective score for atraining subject, and can be trained on a comparison between a predictedlabel (determined by comparing the objective score for the trainingsubjects within a subject comparison group) and the actual label. In afourth variant, the model can be trained to predict the label (e.g.,rating).

The rating model functions to determine a rating for each subject basedon the associated labels (e.g., rank). The system can include one ormore rating models. The rating model can predict the rating, calculatethe rating (e.g., using a rating algorithm), and/or otherwise determinethe rating. Examples of rating algorithms that can be used include: theElo rating algorithm, Gliko rating algorithm, Harkness rating algorithm,and/or any other suitable rating algorithm.

The discretization model functions to segment the subject populationinto discrete groups. The discretization model can be used to generatethe training targets for the scoring model, to discretize the outputs ofthe scoring model, and/or otherwise used. The discretization model candiscretize the training subjects by rank, by rating, and/or otherwisediscretize the subject population. The system can include one or morediscretization models (e.g., for different customers, for training datageneration vs. runtime, etc.). When used in both training and runtime,the discretization model used during training is preferably the same asthat used during runtime, but can alternatively be different (e.g., theruntime model can be specific to an end user or customer). Thediscretization model can be a binning model, clustering model,classification model (e.g., categorization model), and/or any othermodel. The discretization model can use: rules (e.g., ratings 100-500are in bin 1, 500-3000 are in bin 2, etc.); similarity scores (e.g.,rating differences, cosine scores, etc.), statistical binning (e.g., tobin in one or more dimensions; k-means clustering, quantile assignment,etc.), pattern recognition, and/or any other suitable methodology.

The method is preferably performed by a computing system (e.g.,platform), but can additionally and/or alternatively be performed by anyother suitable system.

The computing system can include a remote computing system (e.g. one ormore servers or processing systems); a local system, such as a userdevice (e.g., smartphone, laptop, desktop, etc.); a distributed system;a datastore; a user interface; and/or another computing system. Externalsystems (e.g., user devices, third party systems, etc.) can interactwith the computing system using: an application programming interface(e.g., an API), via a set of requests, via a graphical user interface,via a set of webhooks or events, and/or via any other suitable computinginterface.

However, the system can include any other additional or alternativesuitable components.

4.1. Determining a Subject Comparison Group S100

Determining a subject comparison group S100 functions to determine asubject comparison group (e.g., a group of properties) for training.S100 can be repeated one or more times to obtain one subject comparisongroup, multiple subject comparison groups, a predetermined number ofsubject comparison groups, all possible unique subject comparison groups(e.g., for a given subject set), and/or any other suitable number ofsubject comparison groups. The one or more subject comparison groups arepreferably determined from a training subject set, but can additionallyand/or alternatively not be determined from a training subject set. Thetraining subject set can be limited by one or more subject attributevalues (e.g., only include single family homes, only include subjectsfrom a single neighborhood), be limited to subjects for which labelshave been or can be determined, be otherwise limited, or can beunlimited. Multiple subject comparison groups can be determined from asubject set using sampling techniques with replacement (e.g., whereinthe subject is replaced into the subject set after sampling; wherein asingle subject can be assigned to multiple subject comparison groups),or without replacement (e.g., wherein the subject is not replaced intothe subject set after sampling; wherein a single subject is assigned toa single subject comparison group).

In a first variant, each subject comparison group has a size S, andsubject comparison groups are determined by creating every uniquecombination of S subjects within a subject set (e.g., subject pairs aredetermined based on exhaustive pairing between all subjects within theset).

In a second variant, the subjects for a subject comparison group arerandomly selected from a set of subjects (e.g., with replacement,without replacement, etc.).

In a third variant, unique subject comparison groups are created fromthe subject set. For example, subject comparison groups are created froma set of subjects using a search method (e.g., binary search, Gaussiansearch, etc.).

In a fourth variant, a subject comparison group is manually specified(e.g., comparisons for a real estate appraisal).

In a fifth variant, subject comparison groups are determined to limitthe number of groups needed to train a model that predicts subjects'objective scores. In a first example, an initial subset of subjectcomparison groups are determined (e.g., randomly, according to aheuristic, etc.) and sorted into pools, wherein the remainder of subjectcomparison groups are determined based on the outcomes of the pools. Ina second example, comparison groups are determined using active learningto select comparison groups.

In a sixth variant, subject comparison groups are determined based onsubject attribute values and/or other subject information. In a firstexample, subject comparison groups are determined such that subjectsshare one or more subject attribute values (e.g., same number of bedsand baths, same property class, are comparable, etc.). In a secondexample, subject comparison groups are determined such that there isvariety in subject attribute values within a group and/or across groups.

In a seventh variant, subject comparison groups are determined based onthe output of S300 (e.g., to minimize the total number of pairs).

Optionally, only a subset of the subject comparison groups determined byany of the variants described can be used for further portions of themethod.

However, a subject comparison group can be otherwise determined.

4.2. Determining Subject Information for the Subject Comparison GroupS200

Determining subject information for the subject comparison group S200functions to determine a set of subject information (e.g., a measurementset) for each subject of the subject comparison group. In variants, thesubject information set can be used to determine a label for the subjectcomparison group in S300, as input to the model in S400 (e.g., used astraining data), and/or otherwise used.

The subject information for each subject of the subject comparison grouppreferably shares one or more measurement parameter values (e.g.,associated with the measurement context), but can additionally and/oralternatively not share any measurement parameter values. In a firstexample, subject information can include imagery of properties, and allimages can be taken from the same pose relative to each property ofinterest (e.g., street view image, aerial image, etc.). Subjectinformation can be retrieved from a database, retrieved from a realestate listing service (e.g., MLS™, Redfin™, etc.), and/or otherwisedetermined.

Each subject information set (e.g., measurement set) for a subjectpreferably includes one or more measurements of the subject, but canadditionally and/or alternatively include measurement parameter values,subject attribute values, and/or any other suitable attribute. In afirst example, measurements for the subject comparison group include animage for each subject. In a second example, measurements for thesubject comparison group include multiple measurements (e.g., image,video, audio, digital surface model, etc.) for each subject. In a thirdexample, each measurement set includes subject attribute values (e.g.,square footage, number of beds and baths, etc.), front elevation views,and location information for each subject. In a fourth example, subjectinformation for the subject comparison group can include a combinationof measurements and attributes. In a fifth example, subject informationfor the subject comparison group can include a combination ofmeasurements and measurement parameter values (e.g., scene class,perspective relative to the subject, pose relative to the subject,modality, season, measurement quality, measurement date, measurementtime, measurement location, etc.).

S200 can be repeated for one or more subject comparison groups (e.g.,determined in S100) and/or otherwise repeated.

The type of subject information determined in S200 and used by S300 canbe the same as the type of subject information determined in S500, oralternatively can be different. In a first example, the method caninclude determining neighborhood accessibility scores S200, determininga relative ranking of neighborhoods based on accessibility scores S300,and training the model to predict which neighborhood is better based oncurbside imagery using the accessibility-score-based neighborhoodranking as the training target S400, wherein the test measurement ofS500 includes curbside imagery.

However, measurements for the subject comparison group can be otherwisedetermined.

4.3. Determining a Label for the Subject Comparison Group S300

Determining a label for the subject comparison group S300 can functionto determine a ground-truth label (e.g., comparative label, subjectivelabel, etc.) specifying the subjective comparison between measurementsof the measurement sets of the subject comparison group (e.g., whichhouse is more appealing, which scene is gloomier, which neighborhood isbetter, which roof looks older, which garden is more relaxing, whichinterior is more attractive, which viewshed is more appealing, whichroof is in better condition, etc.). S300 can be repeated for one or moresubject comparison groups (e.g., determined in S100) and/or otherwiserepeated.

One label for each subject comparison group (e.g., subject 1 wins) ispreferably determined, but additionally and/or alternatively one labelfor each subject of the subject comparison group (e.g., subject 1preferred and subject 2 not preferred, subject B wins and subjects A andC lose, an ordered ranking for 3 or more subjects, etc.) can bedetermined, and/or multiple labels for each subject comparison group canbe determined (e.g., wherein an aggregate label is determined from themultiple labels). Each label is preferably binary (e.g., wins and loses,0 and 1, etc.), but can alternatively be non-binary (e.g., multi-subjectranking). Each label can be a numerical label (e.g., 0, 1, etc.), acategorical label (e.g., wins, loses, ties, more appealing, lessappealing, better, much better, etc.), and/or any other suitable labeltype. Each label preferably is represented as a pairwise comparison or agroupwise comparison, but can additionally and/or alternativelyrepresent a score.

Preferably each label is determined based on a measurement (e.g., image)of the subject, but additionally or alternatively can be determinedbased on any other subject information. Each label can be determinedbased on all pieces of information in each subject's information set,one piece of information from each subject's information set, multiplepieces of information from each subject's information set, and/or anyother set of subject information. Each label determined using a subsetof subject information (e.g., one measurement, multiple measurements,etc.) is preferably inherited by the remainder of the subjectinformation set, but can alternatively not be inherited by the remainderof the subject information set. For example, when a measurement setincludes a digital surface model and an image and the label isdetermined based on the image (e.g., subset of the measurement set), thedigital surface model also inherits the determined label.

Preferably labels are manually determined, but alternatively can beautomatically determined (e.g., based on sale price, heuristics, etc.).Each label can be determined by a human vote (e.g., wherein a humanmanually ranks at least one subject of a subject comparison group), byan average of human votes, by a model and/or algorithm, and/or otherwisedetermined. In a first variant, each label is manually determined. Inthis variant, a human labeler is presented with subject information(e.g., measurements) for each of the subjects in a subject comparisongroup. The human labeler is prompted to select their preferred subjectand/or order their subject preferences based on the presentedmeasurements (e.g., example shown in FIG. 5 , by clicking themeasurement, by typing an input, by clicking a display associated withtheir preferred subject, etc.). The selected label is received from theuser. For example, a human labeler can be presented with a firstmeasurement of a first subject (e.g., of an abandoned house with damagedroof and dull exterior) and a second measurement of a second subject(e.g., of a newly renovated home with freshly painted bright exterior).In a first specific example, the human labeler labels the subjectivecomparison between the subject pair and/or measurement pair as “subject2 wins” and/or “second measurement wins.” In a second specific example,the human labeler labels the subjective characteristic of the subject inthe first measurement as “loses” and the subjective characteristic ofthe subject in the second measurement as “wins.” In a third specificexample, the human labeler clicks a specific area of a screen associatedwith their preferred subject to label that subject as the winner. Labelsfor one or more subject comparison groups can be received from the samehuman labeler and/or from multiple human labelers.

In a second variant, the label for subject comparison group can be oneor more ratings for individual subjects. The label for a subject can becalculated based on the results of one or more comparisons involving thesubject (e.g., using the ELO rating formula).

In a third variant, the label for a subject comparison group isautomatically determined by a trained comparison model (e.g.,classifier, neural network, etc.). In this variant, the comparison modelcan determine a label for the subject pair based on the respectivemeasurement sets. For example, S300 can include: extractingrepresentations for the first and second subjects (e.g., vectors) fromthe respective measurement sets (e.g., from the respective images) usinga representation model (e.g., encoder, neural network, etc.); andpredicting a label based on the first and second representations usingthe comparison model (example shown in FIG. 5 ). In a second example,the comparison model can predict the label based on subject attributevalues (e.g., property price, viewshed, etc.) in addition to therepresentation. However, the comparison model can determine the label inany other manner.

The comparison model can be trained on manual labels (e.g., determinedusing the first variation) and/or other preference signals (e.g.,relative number of views, clicks, or dwell time on a real estate listingsite). For example, the comparison model can predict an estimated labelbased on the measurement sets or representations thereof for analready-labeled subject pair, and be trained based on a comparisonbetween the actual label and the estimated label for said subject pair.However, the comparison model can be otherwise trained. When thecomparison model is trained using manually labeled subject pairs, themanually labeled subject pairs are preferably selected from the samesubject set as that labeled by the comparison model, but canalternatively be selected from a different set. In this embodiment, onlya subset of all subject pairs are manually labeled, wherein thecomparison model labels the remainder of the subject pairs. However, thecomparison model can be used to label any proportion of subject pairs.

However, a label for the subject pair can be otherwise determined.

4.4. Training a Model Based on the Label S400

Training a model based on the label S400 functions to train a model thatcan determine an objective score for a subjective characteristic of asubject. The model can be trained once, periodically (e.g., daily,weekly, monthly, yearly, etc.), at random times, when new subject datais available, and/or any other suitable time frequency. In variants,S400 can include determining ratings for the subjects. The model candetermine the objective score based on: measurements of the subject(e.g., elevation views, perspective views, aerial views, etc.), subjectattribute values (e.g., neighborhood, square footage, number ofbeds/baths, etc.), measurement parameter values, and/or any othersubject information. Model inputs can be information relevant to one ormore subjects.

The subject information type used as input for the model can be the sameor different from the subject information type used in S300. In a firstexample, the model can be trained on images taken from the same poseand/or perspective relative to a building as a set of images shown tohuman labelers to determine subject comparison group labels in S300. Ina second example, attribute values can be used to determine the label inS300, and images can be used as input for model training S400.

One or more models can be trained. For example, different models can betrained for different measurement perspectives (e.g., interior,exterior, curbside, top-down, etc.), different subject types, differentgeographic regions, different subjective characteristics, differentsubject information types, and/or other subject parameters.Alternatively, a single model can be trained.

The model can output a score (e.g., a rating) for a subject, aclassification of which subject wins, a relative ranking within thesubject comparison group, which subject will be preferred (e.g., giventhe measurements for both subjects), and/or any other suitable output(e.g., label). In use, the trained model can predict a score for asubject, given a subject information, such as a measurement of thesubject, descriptions of the subject, and/or other information. Themodel can predict, infer, calculate, select, and/or otherwise determinethe output. In variants where a label (e.g., rating) is calculated(e.g., in S300), the rating can optionally be normalized to apredetermined scale (e.g., a scale of 1-10), or remain unnormalized. Inthese variants, the model can be trained to predict the normalized scoreor be trained to predict the unnormalized score (e.g., wherein the scorecan be subsequently normalized before presentation to the user).

In a first variant, S400 can include determining a first and secondscore using a model (e.g., deep learning regression model) based on afirst and second measurement, respectively, of the subject pair,determining a predicted label for the measurement pair based on thecomparison (e.g., determining which subject is preferred based on therespective score), and training the model based on a comparison betweenthe predicted label and the actual label (e.g., ground-truth labelmanually determined previously in S300); example shown in FIG. 3 . Thepredicted label can be calculated based on the first and second scoreusing a ranking system (e.g., Elo rating system, Harkness system, Glickorating system, etc.), by a direct comparison of the scores, bycalculating the probability of winning based on the scores, and/orotherwise determined. In an illustrative example, S400 can includeextracting a first and second vector using an encoder based on a firstand second measurement, respectively, of a first and second subject froma comparison group; predicting scores (ratings) for each subject using amodel; determining a predicted label (e.g., which subject wins/loses/ispreferred/etc.) based on the predicted scores; and training the modelbased on a comparison between the predicted label and the actual labelfor the comparison group.

In a second variant, S400 can include calculating a score for eachsubject of the subject set based on the labels for each subject pair,and training the model to predict the subject's score based on thesubject's measurement set (e.g., the same or different measurement setas that used to predict the label); examples shown in FIG. 4 and FIG. 5. The score can be a rating, rank, and/or other objective measure. Thescore can be calculated using a scoring model leveraging: the Elo ratingsystem, the Harkness rating system, the Glicko rating system, rankingthe subjects in a ranked list based on the labels (e.g., win/loselabels), and/or any other rating or scoring method.

In a third variant (e.g., a binning variant), S400 can include: using afirst model (e.g., the trained model of the first or second variant) tooutput the scores for a plurality of subject measurements; determining adistribution of the scores; binning subsets of the distribution intobins (e.g., 1-10, quantiles, etc.) and/or otherwise normalizing thedata; and training a second model to predict the bin for a subjectmeasurement.

However, the model can be otherwise trained.

In a first illustrative example, generating a trained model includes:determining subject pairs from a subject set; determining labels (e.g.,win/loss labels) for each subject pair; optionally determining a rankedlist of subjects based on the manual labels (e.g., based on whichsubject within the pair was preferred); calculating a score (e.g.,rating) for each subject based on the ranked list and/or manual labelsusing a scoring model; and training a model to predict the score foreach subject; example shown in FIG. 4 . The labels for each subject paircan be determined manually, using a second model (e.g., trained topredict which subject would win/lose based on the subject measurements),and/or otherwise determined.

In a second illustrative example, generating a trained model includes:determining subject pairs from a subject set; determining manual labels(e.g., win/loss labels) for a subset of the subject pairs; training acomparison model to predict the label (e.g., win/loss label or whichsubject wins) based on the manual labels and measurements for thesubject pair subset; labeling the remainder of the subject pairs usingthe comparison model; optionally generating a ranked list based on thelabels; calculating a score (e.g., rating) for each subject based on theranked list using a scoring model; and training a model to predict thescore for each subject; example shown in FIG. 5 .

In a third illustrative example, generating a trained model includes:training the model to predict a first and second score based on thefirst and second measurement, respectively; predicting the label (e.g.,which subject would win) based on the respective scores; and comparingthe predicted label to an actual label (e.g., determined manually orautomatically) for the subject pair; example shown in FIG. 3 . Inexamples, the model can predict the first and second score based on afirst and second vector extracted from the first and second measurement,respectively (e.g., using an encoder); however, the model can otherwisepredict the first and second scores.

In a fourth illustrative example, generating a trained model caninclude: determining a rating for each subject (e.g., based onmanually-determined rankings between subjects, based on the subjects'respective measurements; using the first illustrative example);discretizing the subjects into discrete values (e.g., bins or clusters)based on their ratings; and training a model to predict a subject'sdiscrete value given the subject's measurements. Alternatively, themodel can be trained to predict the subject's rating, wherein the ratingis then discretized.

However, the model can be otherwise trained.

4.5. Generating Synthetic Training Data S425

The method can optionally include generating synthetic training databased on the labels S425, which can function to increase the amount ofdata (e.g., to a statistically significant amount of training data,training data for subjects where the information used to determine thelabel in S300 doesn't exist, etc.) available for training the model.S425 can be performed using the trained model of S400, and/or otherwiseperformed. S425 can be performed after S400, before S400, and/or at anyother time. In an example, S425 can be performed for the binning variantto generate enough scored subjects to determine a statisticallysignificant distribution.

Synthetic training data can include a label for a subject and the typeof subject information used to determine the label, and/or otherinformation. Generating synthetic training data can use GAN, bootstrap,generative models, diffusion models, agent based modeling (ABM), and/orother algorithms and/or techniques.

4.6. Determining a Test Subject S450

Determining a test subject S450 functions to determine a subject to beanalyzed (e.g., property to be analyzed, property of interest, etc.).The subject is preferably determined from a request received from a user(e.g., via an interface, an API, etc.), but can alternatively bedetermined from a geographic region of interest, be depicted within ameasurement (e.g., a wide-scale image), be automatically selected (e.g.,randomly selected, selected based on a set of filters provided by auser, etc.), and/or otherwise determined. The subject can be received asa standalone subject, received as part of a set of subjects, and/orotherwise received. The set of subjects can be determined from: a listof subjects, subjects within a geographic region, subjects (e.g.,properties) currently on the market, subjects satisfying a set ofconditions, subjects depicted within an image, subjects in a list,and/or otherwise determined. Each subject within the set can beidentified by its: address, geolocation, parcel number, lot number,block number, unique ID, and/or any other subject identifier.

However, the subject can be otherwise determined.

The method can additionally or alternatively include selecting a model,which functions to determine how the subject score will be determined.The model can be selected based on the subject's parameters (e.g.,building type, etc.), the subject's available information (e.g., whetherimages are available, whether descriptions are available, etc.), basedon the subject's subjective characteristics to score (e.g., specified bya user, by an API request, etc.), and/or otherwise selected.

4.7. Determining Test Information for the Test Subject S500

Determining test information for the test subject S500 functions todetermine information representative of the subject to be evaluated forits subject characteristic. The test information preferably includes atest measurement, but can additionally or alternatively include subjectattributes (e.g., roof condition, yard condition, roof area, etc.),subject descriptions, and/or other information. The test measurement canbe one test measurement, multiple test measurements, and/or any othersuitable set of test measurements. The test information can havesubstantially the same information parameters (e.g., type, quality,perspective, etc.) and/or different information parameters as those usedto train the model in S400. The test information can have substantiallythe same information parameters and/or different information parametersas those used to determine labels in S300. The test information can beretrieved from a database, retrieved from a real estate listing service(e.g., MLS™, Redfin™, etc.), received as part of a request, and/orotherwise determined.

However, test information for a subject can be otherwise determined.

4.8. Determining a Score for the Test Subject S600

Determining a score for the test subject S600 functions to determine anobjective score of the subjective characteristic of a subject from atest measurement. The score is preferably a numerical score (e.g., 100,500, 2500, etc.), but can alternatively be a categorical score (e.g.,very unappealing, unappealing, neither unappealing nor appealing,appealing, very appealing, etc.). The score is preferably determinedbased on the test information determined in S500, but can be determinedusing any other suitable information.

In a first variant, a score for the test subject can be determined byinputting the test information (e.g., a set of test measurements) intothe trained model, trained in S400 (e.g., example shown in FIG. 9 ). Thescore can optionally be normalized to a predetermined scale (e.g., ascale of 1-10).

In a second variant, a score can be determined using the second or thirdillustrative examples of S400, where the test subject is paired withanother measurement from the set. In an illustrative example, the modelcan predict how the test subject would rank relative to another subjectwithin a subject set based on the subjects' information. The resultantranking can then be used as the score, be used to determine a rating, orbe otherwise used.

However, the score can be otherwise calculated.

S600 can additionally include providing the score to an endpoint throughan interface. The endpoint can be: an endpoint on a network, a customerendpoint, a user endpoint, an automated valuation model system, a realestate listing service (e.g., Redfin™, MLS™, etc.), an insurance system,and/or any other suitable endpoint. The interface can be: a mobileapplication, a web application, a desktop application, an API, adatabase, and/or any other suitable interface executing on a userdevice, gateway, and/or any other computing system. For example, a realestate listing service can display the score (or a normalized version ofthe score) alongside the listing for the subject; example shown in FIG.11 .

However, a score for the test subject can be otherwise determined orused.

5. Use Cases

The method can optionally include using the score for downstreamassessments, which can function to determine one or more values based onthe output of the trained model. The scores can be used with: realestate property investing (e.g., provide a curb appeal score; provide aninterior appeal score; identify underpriced properties that can increasein value through renovation and/or repairs; etc.); identifygentrification of a neighborhood (e.g., average score is increasing fora neighborhood over time); incorporate the score into a valuation model(e.g., to establish the property price; to correct for valuation errors;etc.); identify properties in portfolio that have suffered damage (e.g.,score decreases above a certain threshold); identify properties in aportfolio that need to be reassessed (e.g., score changed above athreshold); identify remodeling options (e.g., based on what istrending/popular/preferred at the time; based on other remodeledproperties' scores; based on a comparison between the scores of propertyremodeling options; etc.); real estate valuations (e.g., use score as aninput to an automated valuation model; use score to detect error inproperty evaluation models; use score as a supplement to aproperty-level valuation report; etc.); real estate and loan trading(e.g., identify deterioration since prior due diligence was completed;determining liquidity scores and/or timelines; incorporate score intocollateral valuation in mortgage origination and in secondary mortgagemarket; etc.); insurance underwriting (e.g., determine pricing ofinsurance depending on the score; optimize inspection to identify whereto send inspectors; determine when to reach out to adjust insurancepolicy, such as when the score changes above a certain threshold;identify which properties to initiate claims for; etc.); roof conditionrating (e.g., wherein the subjective characteristic is roof condition);and/or otherwise used. In an illustrative example, a proposed propertymodification (e.g., remodeling) can be evaluated by generatingmeasurements of the modified property (e.g., synthetic measurements,in-silico measurements, etc.), scoring the modified property based onthe generated measurements using the trained model, comparing the scoreto other proposed modification options (and/or the original property),and optionally selecting the modification with the most desirable (e.g.,highest) score.

However, the score can be otherwise used.

The method can optionally include determining interpretability and/orexplainability of the trained model and/or the resultant score, whereinthe identified attributes (and/or values thereof) can be provided to auser, used to identify errors in the data, used to identify ways ofimproving the model, and/or otherwise used. The method can includedetermining the contribution of at least one attribute of a subject(e.g., property) captured in the subject information (e.g.,measurements) to the objective score. In a first specific example, themethod can be used to determine a set of factors that affect theobjective score (e.g., curb appeal). In a second specific example, themethod can be used to determine an effect of seasonality on propertyvaluation. Interpretability and/or explainability methods can include:local interpretable model-agnostic explanations (LIME), Shapley Additiveexplanations (SHAP), Ancors, DeepLift, Layer-Wise Relevance Propagation,contrastive explanations method (CEM), counterfactual explanation,Protodash, Permutation importance (PIMP), L2X, partial dependence plots(PDPs), individual conditional expectation (ICE) plots, accumulatedlocal effect (ALE) plots, Local Interpretable Visual Explanations(LIVE), breakDown, ProfWeight, Supersparse Linear Integer Models (SLIM),generalized additive models with pairwise interactions (GA2Ms), BooleanRule Column Generation, Generalized Linear Rule Models, TeachingExplanations for Decisions (TED), and/or any other suitable methodand/or approach.

All or a portion of the models discussed above can be debiased (e.g., toprotect disadvantaged demographic segments against social bias, toensure fair allocation of resources, etc.), such as by adjusting thetraining data, adjusting the model itself, adjusting the trainingmethods, and/or otherwise debiased. Methods used to debias the trainingdata and/or model can include: disparate impact testing, datapre-processing techniques (e.g., suppression, massaging the dataset,apply different weights to instances of the dataset), adversarialdebiasing, Reject Option based Classification (ROC),Discrimination-Aware Ensemble (DAE), temporal modeling, continuousmeasurement, converging to an optimal fair allocation, feedback loops,strategic manipulation, regulating conditional probability distributionof disadvantaged sensitive attribute values, decreasing the probabilityof the favored sensitive attribute values, training a different modelfor every sensitive attribute value, and/or any other suitable methodand/or approach.

Different processes and/or elements discussed above can be performed andcontrolled by the same or different entities. In the latter variants,different entities can communicate via: APIs (e.g., using API requestsand responses, API keys, etc.), requests, and/or other communicationchannels.

As used herein, “substantially” or other words of approximation can bewithin a predetermined error threshold or tolerance (e.g., within 0.1%,1%, etc.) of a metric, component, or other reference, and/or beotherwise interpreted.

Alternative embodiments implement the above methods in non-transitorycomputer-readable media, storing computer-readable instructions that,when executed by a processing system, cause the processing system toperform the method(s) discussed herein. The instructions can be executedby computer-executable components integrated with the computer-readablemedium and/or processing system. The computer-readable medium mayinclude any suitable computer readable media such as RAMs, ROMs, flashmemory, EEPROMs, optical devices (CD or DVD), hard drives, floppydrives, non-transitory computer readable media, or any suitable device.The computer-executable component can include a computing system and/orprocessing system (e.g., including one or more collocated ordistributed, remote or local processors) connected to the non-transitorycomputer-readable medium, such as CPUs, GPUs, TPUS, microprocessors, orASICs, but the instructions can alternatively or additionally beexecuted by any suitable dedicated hardware device.

Embodiments of the system and/or method can include every combinationand permutation of the various system components and the various methodprocesses, wherein one or more instances of the method and/or processesdescribed herein can be performed asynchronously (e.g., sequentially),contemporaneously (e.g., concurrently, in parallel, etc.), or in anyother suitable order by and/or using one or more instances of thesystems, elements, and/or entities described herein. Components and/orprocesses of the following system and/or method can be used with, inaddition to, in lieu of, or otherwise integrated with all or a portionof the systems and/or methods disclosed in the applications mentionedabove, each of which are incorporated in their entirety by thisreference.

As a person skilled in the art will recognize from the previous detaileddescription and from the figures and claims, modifications and changescan be made to the preferred embodiments of the invention withoutdeparting from the scope of this invention defined in the followingclaims.

We claim:
 1. A method comprising: determining a property; determining a set of measurements of the property; and determining an objective score for a subjective characteristic of the property based on the set of measurements using a model trained using a set of training properties ranked based on the subjective characteristic.
 2. The method of claim 1, wherein the training properties are manually ranked based on the subjective characteristic; wherein the model is trained by: determining an objective score for each training property based on the respective ranking; and training the model to predict the objective score, based on a set of measurements for the respective training property.
 3. The method of claim 2, wherein the sets of measurements for each training property have substantially the same measurement quality.
 4. The method of claim 2, wherein the sets of measurements for each training property and the property are sampled from the same poses relative to the respective property.
 5. The method of claim 1, wherein the objective score for each training property is determined based on a rating for the training property, wherein the rating is determined from the respective ranking using a rating model.
 6. The method of claim 1, wherein the objective score is a categorical variable value
 7. The method of claim 6, wherein the model is trained by: determining a categorical variable value for each training property based on the respective rank of the training property within the set; and training the model to predict the categorical variable value based on a set of measurements for the training property.
 8. The method of claim 1, wherein the method further comprises determining a property valuation based on the objective score.
 9. The method of claim 1, wherein the method further comprises providing the objective score and at least one image of the property to a user.
 10. The method of claim 1, wherein the set of measurements comprises an oblique image of the property.
 11. The method of claim 1, wherein the set of measurements comprises an interior image of a building of the property.
 12. The method of claim 1, wherein the subjective characteristic comprises curb appeal.
 13. A system, comprising: a storage device; and a set of processors coupled to the storage device, the storage device storing software instructions for controlling the set of processors that, when executed, cause the set of processors to: determine an image of a property of interest; and determine an appeal score for the property based on the image using a scoring model trained on a set of training properties ranked by appeal.
 14. The system of claim 13, wherein the scoring model is trained to predict an appeal score indicative of the ranking for each of the set of training properties.
 15. The system of claim 13, wherein training the model comprises: determining a rating for each training property based on the respective appeal ranking; and training the model to predict an appeal score indicative of the rating based on an image of the respective training property.
 16. The system of claim 13, wherein the ranks for the set of training properties are manually determined by displaying images of two training properties to a user and receiving a user preference between the two training properties.
 17. The system of claim 13, wherein the ranks for the set of training properties are determined based on images of the training properties, wherein all images have substantially a same image quality and each image is taken from substantially a same pose relative to the respective property.
 18. The system of claim 13, wherein the system is further configured to return at least one of the appeal scores or an image of the property of interest.
 19. The system of claim 13, wherein the system is further configured to determine a property modification for the property of interest based on the appeal score.
 20. The system of claim 13, wherein the system is further configured to determine a property valuation for the property of interest based on the appeal score. 